Skip to content

Update Helix Job Monitor and turn it on for the runtime pipeline - #131969

Open
mmitche wants to merge 1 commit into
mainfrom
mmitche-patch-1
Open

Update Helix Job Monitor and turn it on for the runtime pipeline#131969
mmitche wants to merge 1 commit into
mainfrom
mmitche-patch-1

Conversation

@mmitche

@mmitche mmitche commented Aug 6, 2026

Copy link
Copy Markdown
Member

Summary

  • update Microsoft.DotNet.Helix.JobMonitor to 11.0.0-beta.26424.2 from Arcade build 20260824.2
  • pick up the scalable result-processing rewrite from Rewrite the Helix Job Monitor for scalable result processing arcade#17331, including build-scoped discovery, parallel incremental result uploads, retry-aware state, rate-limit handling, and final drain/status reporting
  • add the template's configurable result-upload parallelism and run runtime CI with parallelism 48
  • enable the monitor in the runtime pipeline while allowing path-filtered builds with no Helix submissions

Copilot AI lite review requested due to automatic review settings August 6, 2026 21:12
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the pinned .NET local tool version for microsoft.dotnet.helix.jobmonitor to a newer 11.0.0-beta build, so the repo’s dotnet tool restore / Helix job monitor usage picks up the intended fix.

Changes:

  • Bump microsoft.dotnet.helix.jobmonitor from 11.0.0-beta.26404.8 to 11.0.0-beta.26406.8 in the local tool manifest.

@tarekgh

tarekgh commented Aug 6, 2026

Copy link
Copy Markdown
Member

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-meta
See info in area-owners.md if you want to be subscribed.

Copilot AI review requested due to automatic review settings August 7, 2026 15:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.config/dotnet-tools.json:31

  • The Helix Job Monitor version bump is only applied to the local tool manifest here, but the repo still pins Microsoft.DotNet.Helix.JobMonitor to 11.0.0-beta.26404.8 in eng/Version.Details.xml and eng/Version.Details.props (similar tools like XHarness are kept in sync). This leaves two different "source of truth" versions in the repo and can cause confusion / dependency-flow churn; please update the Arcade dependency pins as well (or explain why this tool is intentionally diverging from Version.Details).
    "microsoft.dotnet.helix.jobmonitor": {
      "version": "11.0.0-beta.26406.8",
      "commands": [

Comment thread .config/dotnet-tools.json Outdated
Copilot AI review requested due to automatic review settings August 7, 2026 22:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.config/dotnet-tools.json:31

  • The PR title indicates updating helix.jobmonitor to 11.0.0-beta.26406.8, but the tool manifest pins 11.0.0-beta.26407.3. Please align the pinned version with the intended update (or update the PR title/description to match) to avoid confusion when tracking tool rollouts.
    "microsoft.dotnet.helix.jobmonitor": {
      "version": "11.0.0-beta.26407.3",
      "commands": [

Copilot AI review requested due to automatic review settings August 11, 2026 14:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.config/dotnet-tools.json:31

  • The PR title says the helix.jobmonitor version is being updated to 11.0.0-beta.26406.8, but the tool manifest is actually updated to 11.0.0-beta.26411.1. Please align these (either update the PR title/description to match the version being shipped, or change the version here if 26406.8 is the intended target).
    "microsoft.dotnet.helix.jobmonitor": {
      "version": "11.0.0-beta.26411.1",
      "commands": [

Copilot AI review requested due to automatic review settings August 11, 2026 15:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.config/dotnet-tools.json:31

  • The PR title says the helix.jobmonitor version is being updated to 11.0.0-beta.26406.8, but this change pins microsoft.dotnet.helix.jobmonitor to 11.0.0-beta.26411.1. Please align the PR title/description with the version actually being used, or update the manifest to the intended version.
    "microsoft.dotnet.helix.jobmonitor": {
      "version": "11.0.0-beta.26411.1",
      "commands": [

Copilot AI review requested due to automatic review settings August 11, 2026 19:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.config/dotnet-tools.json:31

  • PR title says update helix.jobmonitor to 11.0.0-beta.26406.8, but the actual change sets it to 11.0.0-beta.26411.1 (from 26407.8). Please align the PR title/description with the version being introduced, or adjust the version here to match the stated target.
    "microsoft.dotnet.helix.jobmonitor": {
      "version": "11.0.0-beta.26411.1",
      "commands": [

Copilot AI review requested due to automatic review settings August 12, 2026 04:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.config/dotnet-tools.json:30

  • The PR title says the Helix job monitor is being updated to 11.0.0-beta.26406.8, but the tool manifest is updated to 11.0.0-beta.31411.1. Please align the PR metadata and the actual version being consumed (either update the title/description or change the manifest version) so it’s clear what build is being taken and why.
      "version": "11.0.0-beta.31411.1",

NuGet.config:15

  • Adding the new 'general-testing' package source affects restores for the entire repo (extra feed to probe and a potentially different source of packages). If this feed is only needed to acquire the updated helix.jobmonitor tool, consider scoping it to just that restore path (or document why the global source addition is required) to avoid unintended restore/source selection changes.
    <add key="general-testing" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json" />

Copilot AI review requested due to automatic review settings August 12, 2026 04:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.config/dotnet-tools.json:31

  • The PR title/description says the update is to helix.jobmonitor 11.0.0-beta.26406.8, but the change here updates to 11.0.0-beta.31411.2 (from 11.0.0-beta.26407.8). Please confirm which version is intended and align either this value or the PR metadata so the change is traceable.
    "microsoft.dotnet.helix.jobmonitor": {
      "version": "11.0.0-beta.31411.2",
      "commands": [

Copilot AI review requested due to automatic review settings August 12, 2026 06:58
Copilot AI review requested due to automatic review settings August 20, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (2)

eng/pipelines/runtime.yml:58

  • enableHelixJobMonitor is defined twice in the same variables: block (first false, then true). In Azure Pipelines the later definition wins, so this effectively forces the monitor on for all runs, and the earlier entry is just confusing/noise.

If the intent is to force-enable the monitor, consider collapsing this to a single definition (and update the comment accordingly) so the effective value is obvious.

  # Keep the monitor enabled in the PR merge even when the target branch
  # temporarily changes the default above.
  - name: enableHelixJobMonitor
    value: true

.config/dotnet-tools.json:31

  • The PR title says this updates helix.jobmonitor to 11.0.0-beta.26406.8, but the tool manifest pins microsoft.dotnet.helix.jobmonitor to 11.0.0-beta.31411.14 (and the previous value was 11.0.0-beta.26411.119, which is already newer than 26406.8).

Please confirm the intended version and update either the PR title/description or the pinned tool version so they match.

    "microsoft.dotnet.helix.jobmonitor": {
      "version": "11.0.0-beta.31411.14",
      "commands": [

Copilot AI review requested due to automatic review settings August 20, 2026 20:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (2)

eng/pipelines/runtime.yml:59

  • The pipeline defines enableHelixJobMonitor twice (first false, then true), so the earlier assignment is overridden and effectively dead. This is confusing and the accompanying comment is misleading; prefer a single definition that clearly overrides the default from common/variables.yml.
  # Keep the monitor enabled in the PR merge even when the target branch
  # temporarily changes the default above.
  - name: enableHelixJobMonitor
    value: true

.config/dotnet-tools.json:30

  • The PR title says this updates microsoft.dotnet.helix.jobmonitor to 11.0.0-beta.26406.8, but the tool manifest is pinned to 11.0.0-beta.31411.15. Please align the PR title/description with the actual version (or adjust the pinned version if the title is the intended target).
      "version": "11.0.0-beta.31411.15",

Copilot AI review requested due to automatic review settings August 21, 2026 00:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (2)

eng/pipelines/runtime.yml:58

  • enableHelixJobMonitor is assigned twice with conflicting values (false then true). Since there are no uses between these assignments, the first assignment is redundant and makes it harder to understand which behavior is intended.
  # Keep the monitor enabled in the PR merge even when the target branch
  # temporarily changes the default above.
  - name: enableHelixJobMonitor
    value: true

.config/dotnet-tools.json:30

  • The PR title/description says the helix job monitor is being updated to 11.0.0-beta.26406.8, but this change pins microsoft.dotnet.helix.jobmonitor to 11.0.0-beta.31411.16. Please reconcile the intended version (or update the PR title/description to match what is actually being used).
      "version": "11.0.0-beta.31411.16",

Copilot AI review requested due to automatic review settings August 21, 2026 14:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (2)

eng/pipelines/runtime.yml:58

  • The variable enableHelixJobMonitor is declared twice (first false, then true). In Azure Pipelines the later value wins, which makes the earlier entry misleading and easy to accidentally “toggle” without effect. Consider keeping a single declaration with the intended value and keep helix-platforms.yml as a separate entry.
  - name: enableHelixJobMonitor
    value: false
  - template: /eng/pipelines/helix-platforms.yml
  # Keep the monitor enabled in the PR merge even when the target branch
  # temporarily changes the default above.
  - name: enableHelixJobMonitor
    value: true

.config/dotnet-tools.json:31

  • The PR title says the job monitor is being updated to 11.0.0-beta.26406.8, but the manifest pins microsoft.dotnet.helix.jobmonitor to 11.0.0-beta.31411.17. Please align the pinned version (and any related feed changes) with the intended version, or update the PR title/description accordingly.
    "microsoft.dotnet.helix.jobmonitor": {
      "version": "11.0.0-beta.31411.17",
      "commands": [

Copilot AI review requested due to automatic review settings August 21, 2026 15:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

eng/pipelines/runtime.yml:58

  • enableHelixJobMonitor is defined twice in the same variables block. The second definition currently overrides the first, making the initial false assignment redundant and potentially confusing (and order-dependent for template evaluation). Prefer a single definition with the effective value.
  # Keep the monitor enabled in the PR merge even when the target branch
  # temporarily changes the default above.
  - name: enableHelixJobMonitor
    value: true

Comment thread .config/dotnet-tools.json Outdated
Copilot AI review requested due to automatic review settings August 24, 2026 20:56
@mmitche mmitche changed the title Update version of helix.jobmonitor to 11.0.0-beta.26406.8 Update Helix Job Monitor to 11.0.0-beta.26424.2 Aug 24, 2026
Use the promoted scalable Job Monitor, add configurable result-upload parallelism, and enable the monitor in the runtime pipeline.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f6ec29d9-df3d-405a-a823-571c6a18e447

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

Comment thread eng/common/core-templates/job/helix-job-monitor.yml
Copilot AI review requested due to automatic review settings August 24, 2026 21:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.

@premun
premun removed the request for review from akoeplinger August 25, 2026 14:19
@premun premun changed the title Update Helix Job Monitor to 11.0.0-beta.26424.2 Update Helix Job Monitor and turn it on for the runtime pipeline Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants